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@ Robotic control system for a needle sorting and feeding apparatus. 

(57) A control system for a needle infeed device 
for automatically transferring needles randomly 
positioned on an indexing conveyor to an en- 
gagement device for subsequent conveyance to 
a processing location. The needle infeed device 
comprises one or more robots each having a 
gripper means for picking and placing the nee- 
dles in the engagement device. The control 
system comprises a control device for pausing 
the indexing conveyor to create a dwell cycle 
for the infeed device, vision tracking apparatus 
in communication with the control device for 
imaging needles at one or more predetermined 
locations on the indexing conveyor and for 
determining positional and orientation data for 
each recognized needle during the dwell cycle, 
a memory device for temporarily storing the 
positional and orientation data received from 
the vision tracking device, and, a robot control _ 
device for accessing the stored positional and 
orientation data corresponding to the imaged CD 
needles from the memory means and for enabl- ~ 
ing one of the robots to pick up the imaged 
needle in accordance with its respective 
positional and orientation data in order to place 
the needle in the engagement device. 








UJ 

_i 


.ACE 


ROB 


PCK 


NEE! 


CD 




Jouve, 18, rue Saint-Denis. 75001 PARIS 



1 



EP 0 667 1 24 A1 



2 



1. Field of the Invention 

The present invention relates generally to ma- 
chines for automatically producing armed surgical 
needles, such as surgeons' needles having sutures 
attached thereto, and more specifically, to a control 
system for a needle infeed apparatus that automati- 
cally transfers randomly positioned surgical needles 
at one location and conveys them to another location 
in an oriented position. 

2. Description of the Prior Art 

Most armed surgical needles, i.e., needles having 
sutures attached to one end thereof, that are in pres- 
ent use by surgeons and medical personnel, are man- 
ufactured utilizing manual and semi-automated pro- 
cedures such as those described in U.S. Patent Nos. 
3,611,551, 3,980,177, and 4,922,904. For instance, 
as described in U.S. Patent No. 3,61 1 ,551 , manual in- 
tervention is required by an operator to accurately 
position a suture within the needle for swaging and to 
adjust swaging dies to increase or decrease swage 
pressure when suture strands of different gauges are 
to be swaged. This process is costly in terms of man- 
hour labor and efficiency because manual positioning 
is required for swaging to take place. 

Presently, suture material may be supplied 
wound on a bobbin, or, a king or driven spool before 
being cut and positioned within the swaging end of a 
surgical needle. In U.S. Patent No. 3,980,177 the su- 
ture material is fed from a spool and taken up on a ro- 
tating tension rack where uniform length strands are 
subsequently cut. Thus, the length of the suture is de- 
termined by the size of the rack and manual interven- 
tion is required to prepare the rack for the cutting of 
the suture material wound thereabout. Moreover, 
manual intervention is required to change the rack 
each time a suture strand of different length is de- 
sired. 

In U.S. Patent No. 4,922,904, the suture material 
is supplied wound on a bobbin and is fed through va- 
rious guide means and a heater for straightening the 
material, prior to insertion within the crimping cavity 
of the surgical needle. In one embodiment shown 
therein, an elaborate television monitoring means is 
required for aligning the drawn suture within the 
crimping cavity of the surgical needle prior to swaging 
thereof. In the same embodiment, a rotary encoder 
device is used to determine the length of suture ma- 
terial unwound from the bobbin prior to cutting. In an 
alternative embodiment, after swaging of the indefin- 
ite length of suture material to the needle, the needle- 
suture assembly is fed a predetermined distance prior 
to cutting to obtain a suture strand of predetermined 
length. Thus, to obtain uniform lengths of suture ma- 
terial every time requires careful manipulations and 
precise controls, and the processes used to accom- 



plish these tasks are also costly in terms of man-hour 
labor and efficiency. 

It would be highly desirable to provide an armed 
needle production and packaging system that is fully 
5 automated and that can automatically feed surgical 
needles for conveyance to an automatic swaging ma- 
chine for the swaging of sutures thereto. 

It would also be highly desirable to provide a nee- 
dle sorting device that can efficiently and accurately 
w orient a needle for subsequent transference to an au- 
tomatic swaging station. 

Even more desirable would be a provision of a ro- 
botic control system to maintain the efficiency and in- 
tegrity of the needle sorting and conveying function. 

15 

SUMMARY OF THE INVENTION 

Accordingly, it is an object of the instant invention 
to provide a control system for an automatic needle 
20 sorting device that conveys properly oriented individ- 
ual needles to a fully automated needle swaging sta- 
tion. 

It is another object of the instant invention to pro- 
vide a cost effective needle sorting device that virtu- 
25 ally eliminates operator exposure to repetitive manual 
operations. 

Furthermore, it is an object of the instant inven- 
tion to provide a robotic control system for an auto- 
matic needle sorting device that maintains the integ- 

30 rity of the needle sorting function, and ensures high- 
speed and efficient conveyance of sorted, oriented 
needles to a fully automated needle swaging station. 

These and other objects of the present invention 
are attained with a control system for a needle infeed 

35 apparatus for automatically transferring needles ran- 
domly positioned on a forwardly indexing conveyor to 
an engagement device for subsequent conveyance to 
an automatic needle swaging station. The needle in- 
feed apparatus comprises one or more robots each 

40 having a gripper device for picking and placing the 
needles. The control system comprises: a control de- 
vice for pausing the indexing conveyor to create a 
dwell cycle for the infeed device; a vision tracking de- 
vice for visually recognizing needles at one or more 

45 predetermined locations on the indexing conveyor 
and for determining positional and orientation data for 
each recognized needle during the dwell cycle; mem- 
ory device for temporarily storing said positional and 
orientation data received from the vision tracking de- 

50 vice; and, a robot control apparatus for accessing the 
stored positional and orientation data corresponding 
to the visually recognized needles from the memory 
device, and enabling one of the robots to pick up the 
recognized needle in accordance with its respective 

55 positional and orientation data and place the needle 
in the engagement device. 

The needle sorting system is provided with a re- 
dundant vision tracking device a second robot assem- 



2 



3 



EP 0 667 124 A1 



4 



bly means that operate in the manner as described 
above. The redundancy is designed in the system is 
to ensure that a continuous and uninterrupted flow of 
60 needles/minute is supplied to the automatic swag- 
ing station. 

Further benefits and advantages of the invention 
will become apparent from a consideration of the fol- 
lowing detailed description given with reference to the 
accompanying drawings, which specify and show 
preferred embodiments of the invention. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a block diagram showing the process 
flow for the needle sorting apparatus of the present 
invention. 

Figure 2 is a surgical needle 1 9 with crimping end 
85 and barrel portion 83. 

Figure 3(a) is a top view of the needle sorting de- 
vice 20 of the instant invention. 

Figure 3(b) is a side elevational view of the needle 
sorting device of Figure 3(a) showing the robot as- 
sembly above the first and second conveyor means 
and the tracking means comprising two video cam- 
eras for tracking the position of individual needles and 
the control system means for processing said data. 

Figure 4 is a detailed side view of the needle in- 
feed means for singulating and depositing needles 
onto a translucent conveyor. 

Figure 5(a) is a detailed view of the precision con- 
veyor boat having jaws for engaging and retaining an 
oriented needle for subsequent swaging. 

Figure 5(b) is a detailed elevation view of the pre- 
cision conveyor boat taken along line 5-5 of the boat 
illustrated in Figure 5(a). 

Figure 5(c) is a detailed view of the precision con- 
veyor boat with movable jaw extended for placement 
of needle oriented for automatic swaging. 

Figure 6 is a side view of the robot load solenoid 
that actuates the jaws of the precision conveyor boat. 

Figure 7 is schematic representation of the con- 
trol and data flow for each of the control tasks of the 
infeed control system of the present invention. 

Figures 8(a) - 8(f) illustrate the flow diagrams for 
the various robot control, vision control, and conveyor 
indexing tasks to be performed by the needle infeed 
control system of the instant invention. 

Figure 9(a) is a side view of the needle rollover 
(plow) which ensures uniform orientation of the nee- 
dle on the conveyor boat prior to automatic swaging. 

Figure 9(b) is a front view of the plow taken along 
line 9-9 of Figure 9(a). 

Figures 9(c)-9(e) is a front view illustrating the 
plow 54 orienting a needle in one direction upon a 
boat 40 of the precision conveyor. 

Figure 1 0(a) is a side view of the needle hard stop 
assembly 95 for further orienting the needle 1 9 within 
the engagement jaws of conveyor boat 40. 



Figure 10(b) is a top plan view of the needle hard 
stop assembly 95 for further orienting the needle 19 
within the engagement jaws of conveyor boat 40. 

Figure 11(a) is a side view of the stop assembly 
5 80 for further orienting the needle 1 9 upon conveyor 
boat 40. 

Figure 11(b) is a front view of the stop assembly 
80 for further orienting the needle 19 upon conveyor 
boat 40 taken along line 11-11 of Fig 11(a). 

10 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENT 

This invention is drawn to a control system for a 
15 needle infeed apparatus that is designed to automat- 
ically sort, singulate, and convey surgical needles of 
various sizes to an automatic swaging station where 
sutures are attached to individual needles. A typical 
surgical needle 19 having a barrel portion 7, an arc- 
20 uate blade portion 8, and a suture receiving end or 
opening 5 for swaging a suture thereto, is illustrated 
in Figure 2. 

Generally, in the needle sorting process 10 
shown in Figure 1, needles are first loaded into a vi- 

25 bratory bowl at step 11, automatically sorted and lin- 
early fed at step 1 2 to a translucent indexing conveyor 
at step 13, evaluated with respect to orientation and 
position by a vision tracking system at step 14, picked 
up by a robot apparatus at step 15, transferred to a 

30 engagement boat of a precision conveyor by the robot 
apparatus at step 16, and finally conveyed to a multi- 
axis indexing means for further conveyance to subse- 
quent swaging workstation at step 17. An explanation 
of the apparatus used to carry out each step will be 

35 explained in detail hereinbelow. A full detailed de- 
scription of the needle sorting apparatus can be 

found in copending patent application 

(attorney docket No. 8920) assigned to the same 

assignee of the present invention. 

40 The preferred embodiment of the needle sorting 

and infeed apparatus 20 is illustrated in the top view 
of the system in Figure 3(a) and the side view of Fig- 
ure 3(b). As shown therein, needles 19 are delivered 
in bulk to each of two vibratory bowls or hoppers 

45 21a,b where they are singulated by respective singu- 
lating assemblies 22a,b and randomly deposited 
upon each of two translucent conveyors 25a,b. The 
two translucent conveyors 25a,b carry the randomly 
deposited needles 19 in the direction indicated by the 

50 arrow in Figure 3(a) where their position and orienta- 
tion are evaluated by a remotely located vision track- 
ing system that will be discussed in detail below with 
respect to Figure 3(b). This tracking system, eval- 
uates the position and orientation of each available 

55 needle upon translucent conveyor 25a as it forwardly 
conveys the needles over illuminated (backlit) plat- 
forms 30a and 30b, and further evaluates the position 
and orientation of the each available needle upon 
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translucent conveyor 25b as it forwardly conveys the 
needles over illuminated (back! it) platforms 33a and 
33b. The orientation and positional information ob- 
tained from the vision tracking system is processed 
and converted to information usable by each of two 
robot assemblies 50a,b for instructing respective ro- 
bot grippers 55a,b to pick up and transfer identified 
needles from one of the translucent conveyors to in- 
dividual engagement boats 40, located on a precision 
conveyor 35 that is also being indexed in the same di- 
rection as the translucent conveyors as shown in Fig- 
ure 3(a). The control system of the invention instructs 
a robot gripper, for e.g., grip per 55a of the robot as- 
sembly 50a, to grab the tracked needle from one of 
the two conveyors 25a,b for a dwell cycle of the sys- 
tem, i.e. when the conveyor has paused. If the ran- 
domly deposited needles 19 are oriented such that 
either of the robot grippers 55a, b is unable to pick one 
of them up or place the needle onto the precision con- 
veyor because of its limited range of motion, the con- 
trol system will execute a recovery procedure to en- 
sure that there are no shortages of needles 19 to be 
fed by the precision conveyor 35 to the automatic 
high-speed swaging workstation (not shown) which 
can achieve up to 60 needle swages per minute. 

In the preferred embodiment, the timing of each 
conveyor 25a,b is identical, but the dwell periods are 
out of phase. Because of the phased timing, the vision 
tracking system will be identifying needles on one in- 
dexing conveyor, for e.g. 25a, while both robots are 
picking needles from the other indexing conveyor 25b 
and placing each needle in an individual engagement 
boat of the precision conveyor. Similarly, while both 
robots are picking needles from the indexing convey- 
or 25a, the vision tracking system will be identifying 
needles on the other indexing conveyor 25b. 

The first step of the automatic swage/wind proc- 
ess 10 involves introducing a predetermined amount 
of needles 19 from an infeed device, such as a bowl 
or hopper, into a needle singulating assembly. In the 
preferred embodiment shown in the side view of Fig- 
ure 4, a vibratory hopper or bowl 21 is provided with 
a suitable optical or mechanical counting device such 
as sensor plate 24 so that up to six (6) needles may 
be periodically fed into the sorting assembly 22 at any 
one time. The needles 19 are dispensed from the vi- 
bratory hopper 21 through a gate 18, and fall by grav- 
ity through a needle singulating assembly 22 which 
comprises a series of di verier doors 23a, b and trap 
doors 23c,d that alternate between two positions to 
allow one half of the needles discharged to drop onto 
each of two spaced-apart chutes 26,28 and finally 
onto the moving translucent indexing conveyors 
25a,b. With the diverter door 23a in the position 
shown in Figure 4, any needle 19 introduced into the 
needle singulating assembly 22 will be deflected and 
fall into an external receptacle (not shown) where the 
needles may be subsequently returned to the hopper 



21 . When diverter door 22a is in a second position in- 
dicated by the dotted line diverter 23a' in Figure 4, up 
to twelve (12) needles 9 will be counted by sensor 44 
as they fall through the sorting assembly so that they 

s may be singulated by appropriate switching of diver- 
ter door 23b and trap doors 23c,d. The reciprocating 
motion of the diverter doors 23a,b and trap doors 
23c,d are timed to ensure that approximately six (6) 
needles are deposited in two lines of three on each 

10 conveyor 25a,b at a time. In the preferred embodi- 
ment, this number of needles, as deposited, will oc- 
cupy approximately eight (8) inches of indexing con- 
veyor length, and thus, ensure that the needles are 
adequately spaced apart when deposited. Preferably, 

15 the diverter doors 23a,b reciprocate under the control 
of an automatic control system and timed to alternate 
between two positions to allow approximately three 
(3) needles at a time to drop onto each translucent in- 
dexing conveyor 25a, b via respective discharge 

20 chutes 26,28. Both diverter doors 23a,b are respec- 
tively driven by cylindrical pistons 27a, b and suitable 
solenoid or pneumatic motors (not shown). It should 
be understood that any needle 1 9 deposited on trans- 
lucent indexing conveyors 25a, b will be randomly 

25 positioned and unoriented. Preferably, each translu- 
cent indexing conveyor 25a, b is an endless belt con- 
veyor that is driven at a rate of four inches per second 
at steady state (4 in Vsec) and runs parallel to the pre- 
cision conveyor 35 as shown in Figures 3(a). 

30 As described above, and in view of Fig. 3(a), the 

robot assembly comprises two robots 50a,b located 
downstream from each needle singulating assembly 
22a,b and proximate both the precision and translu- 
cent indexing conveyors. In the preferred em bod i- 

35 ment described herein, each robot assembly 50a, b is 
an Adept® 604- S robot capable of accomplishing 
needle transfers at a rate of approximately 40 trans- 
fers per minute as controlled by each robot's corre- 
sponding Adept® CC controller. Each robot is a four- 

40 axis SCARA (Selective Compliance Assembly Robot 
Arm) robot comprising four Joints: Joint 1, being the 
shoulder joint having a rotational range of motion of 
+/- 100°; Joint 2, the elbow joint, having a rotational 
range of motion of +/- 140°; Joint 3 providing transla- 

45 tional motion for a robot quill for up to 150mm in an 
up down motion; and, Joint 4, being the wrist joint, 
providing +/- 360° rotational motion of the quill. Robot 
grippers 55a, b are attached to the quill of each re- 
spective robot assembly 50a, b and are enabled to 

so provide gripping action by pressure supplied from an 
air cylinder (not shown). 

Referring now to Figure 3(b), there is illustrated 
the precision conveyor 35 which is driven by drive mo- 
tor assembly 42 at a rate sufficient to index and trans- 

55 fer approximately one oriented surgical needle per 
second (1 needle/sec) to the automatic swaging ma- 
chine. A similar drive motor assembly 43 is provided 
for driving the indexing conveyors 25a,b. As will be 
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explained in detail below, each drive motor assembly 
42,43 is interfaced with and operate under the control 
of the control system 69 to pause the indexing motion 
to enable the pick-up and transfer of a needle from 
the indexing conveyor to the precision conveyor. In 
the preferred embodiment, the control system 69 in- 
cludes a programmable logic controller (PLC) that is 
also in digital communication with the Adepts robot 
controllers and the vision tracking system compo- 
nents to control the infeed system. 

As shown in Figure 3(b), the vision tracking sys- 
tem comprises a camera assembly 60 having two vid- 
eo cameras 62 and 64, one located overhead each re- 
spective illuminated platform portion, 30a and 30b ? 
for its indexing conveyor 25a. As will be explained in 
detail below, the video images of the needles ob- 
tained from each camera 62,64 are bit-mapped or 
suitably digitized and transmitted via suitable trans- 
mission media, such as communication lines 67a,b 
shown in Figure 3(b), to the remotely located control 
system computer 69 where a Vision Control Task 1 60 
processes the video images and inputs the data to 
each robot 50a,b via communication lines 197. Pre- 
ferably, the conveyors 25a and 25b are translucent 
and are backlit at the respective portions 30a,b and 
33a,b so that a sharp video image may be obtained 
by the overhead camera assembly for processing. It 
is understood that for descriptive purposes, only two 
video cameras 62,64 corresponding to the two illu- 
minated platforms 30a, 30b are shown in Figure 3(b). 
However, the invention includes a second set of video 
cameras (not shown) corresponding to illuminated 
platforms 33a and 33b for conveyor 25b so that, as 
mentioned above, binary images of needles on con- 
veyor 25b may be obtained while the robots are pick- 
ing and placing needles from conveyor 25a. The re- 
dundancy designed into this system ensures that 
there will be no momentary shortage of needles fed 
to the swaging station and that maximum throughput 
of oriented needles for input to the swaging station is 
achieved. In the event the state of robotics technology 
improves, and as the robot assemblies achieve great- 
er degrees of movement at faster speeds, the second 
set of cameras and a second robot assembly may no 
longer be required. Furthermore, a robotic assembly 
of sufficient speed and precision may be able to pick 
up randomly deposited needles from a moving con- 
veyor and place them directly in an oriented position 
at the swaging station. 

In the preferred embodiment, each camera 62,64 
is mounted approximately one (1) meter above each 
backlit indexing conveyor 25a, b and utilizes an elec- 
trically controlled telephoto lens with a focal distance 
ranging from 10mm to 140mm that may be changed 
with suitable adaptors. Suitable lens controllers are 
used to establish lighting/iris, focus, and field of view 
for each camera lens, and, are interfaced with the 
Adept® controller via an RS-232 link. 



Af urther component of the control system for the 
needle sorting and infeed apparatus includes a SCA- 
DA Node (not shown) which is used to oversee and di- 
rect the infeed system. This node interfaces with each 

5 of the Adept® controllers via discrete RS-232 links 
which are used to download data information, such as 
needle parameters, error messages, and status mes- 
sages, to the Adept® controllers during run-time. The 
SCADA node may comprise a personal computer or 

w such suitable device, running commercially available 
FIXDMACS® software. Serial communication is used 
to exchange the needle parameters entered at the 
FIX/DMACS "Adept® Setup" screen during a needle 
changeover procedure to be described below. After 

15 an operator enters the needle parameters and ini- 
tiates a changeover, the FIX/DMACS Node will trans- 
mit these parameters to the robot controller(s). 

The robotic/vision control system 69 of the inven- 
tion comprises individual computer software pro- 

20 grams, each associated with a particular task to be 
performed by the needle sorting and infeed system 
1 0 and executed under the control of the PLC 120. A 
flow diagram representation of the tasks to be per- 
formed and the PLC control signals for enabling such 

25 tasks is illustrated in Figures 8(a) - 8(h). As shown in 
Figure 7, the software for the robotic control system 
69 of the instant invention performs eight (8) main 
tasks: a Robot Control task 150; a Vision Control task 
160; a Conveyor Indexing Control task 180; a SCADA 

30 Node Interface task 195; A Control Panel task 260; a 
Task Manager 240; a Conveyor Initiation task 190; 
and, a Lens Control task 195. Of these eight tasks 
mentioned above, the first six are active during the 
needle infeed steady state operation as will be ex- 

35 plained below. Figure 7 additionally shows the data 
flow among the tasks and the signals which initiate 
the tasks. It is understood that the software language 
used in the preferred embodiment, is Adept's V/V+ 
language, which supports both vision and robotic 

40 control in a multitasking environment. 

It should be understood to those skilled in 5 the 
art that each robot assembly, controllers, and camera 
vision tracking system requires careful calibration 
and configuration procedures for the infeed system to 

45 properly function. For instance, each robot assembly 
requires that joint positions be set and joint limits be 
configured to ensure that the robots avoid structural 
damage when enabled. Furthermore, a camera-to-ro- 
bot calibration is required so that the vision system 

50 may accurately compute the positional coordinates of 
the needle so that the robot may move to the pick pos- 
ition. This procedure provides a translation matrix be- 
tween the camera's f ield-of-view and each robot base 
position. 

55 The PLC 120 is responsible for initially powering 

the robot controllers and robots. A robot calibration 
procedure may be initiated after power-up to move 
the robot joints to known "home" positions to syn- 
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chronize the digital encoders (not shown). 

The process of starting the PLC 120, robot con- 
trollers, and conveyors 25a,b and 35 is time-critical. 
From the robot controller perspective, when a RO- 
BOT ENABLE signal 219 is raised by PLC 120, it be- 
gins its normal cycle by executing the Robot Control 
Task 150, the Vision Control Task 160, the Conveyor 
Indexing Control Task 180, and the Conveyor Initia- 
tion Task 190; which initiates the movement of con- 
veyor 25a, waits approximately up to two (2) seconds, 
and then initiates the movement of second conveyor 
25b as will be described in detail below. The PLC si- 
multaneously raises the ROBOT ENABLE signal on 
the other Adept robot. Under this scenario, the PLC 
integrates the startup of the Needle Infeed System, 
the Indexing Conveyors, and 5 swaging machine with 
the raising of the ROBOT ENABLE signal 219. As will 
be explained in further detail below, when the RO- 
BOT ENABLE signal goes low, the Adept robot halts 
its standard processing and responds to requests 
from the SCADA node. 

Robot Control Task 

There is a single Robot Control task associated 
with each Adept® controller for each robot assembly 
50a,b although only one is indicated as element 150 
in Figure 7. The control system software for the Robot 
Control task 150 manages the respective robot as- 
sembly 50a or 50b as a resource, reads a FIFO buffer 
1 55 of identified needle locations which are produced 
by and input from the Vision Control Task 160, inter- 
faces with the programmable logic controller (PLC) 
120 of control system 69 for needle placement hand- 
shaking, and, initiates the indexing of the conveyor 
belts 25a,b. 

As shown in the block diagram of Figures 8(a) - 
8(c), the steady state operation of the Robot Control 
task 150 for each robot assembly 50a, (50b) is as fol- 
lows: 

First, the respective robot controller continuously 
polls its input FIFO 155 via data line 193 to obtain 
positional coordinate data for the identified needle lo- 
cations on a respective translucent conveyor 25a or 
25b as indicated as step 102. The data for the needle 
locations are provided to the FIFO buffer from the Vi- 
sion Control task 160 via respective data lines 197 as 
will be explained in further detail below. When an ac- 
ceptable (recognizable) needle position is entered 
into the FIFO buffer 155, the robot controller will re- 
move the needle position from the buffer and direct 
the robot gripper arm 55a,(55b) to move to that loca- 
tion on the conveyor belt as indicated at step 104. 
Next, for each recognized needle, the Robot Control 
task 150 will signal the robot gripper 55a,(55b) to 
close on the needle barrel portion 7 and to depart 
from the conveyor, as indicated at step 106, to an ap- 
proach location proximate the precision conveyor 35. 



The robot control task then generates a NEEDLE IN 
GRIPPER signal 207 to the PLC as indicated at step 
108 and waits for a response from the PLC 120. As 
shown as step 109 in Figure 8(a), and, in further view 
5 of Figure 7, when the PLC receives a Robot task gen- 
erated NEEDLE IN GRIPPER signal 207, the PLC 

120 will generate a SAFE TO PLACE signal 191 for 
receipt by each of the robots 50a,b. The purpose of 
the SAFE TO PLACE signal 191 is to inform the re- 

w spective robot assembly 50a,b that a needle may be 
placed onto a precision conveyor boat 40 of conveyor 
35. As a response to the receipt of the SAFE TO 
PLACE signal 191, the Robot Control task 150 will 
generate a DO NT INDEX PRECISION CONVEYOR 

15 signal 204 at step 111 for receipt by the PLC 120 im- 
mediately before it places the needle on the precision 
conveyor 35. While this signal remains high, for e.g., 
at a logic "1" state, the Adepts robot 50a or 50b will 
attempt to place a needle onto a boat 40 of precision 

20 conveyor 35 as indicated as step 113 in Figure 8(b). 
This involves initiating the engagement jaws 47,49 of 
the precision conveyor engagement boat 40 to retract 
to allow the placement of the needle therebetween, 
as will be explained below. Once the movement of the 

25 robot has settled and a needle is placed, the Robot 
task 150 will generate a NEEDLE PLACE COM- 
PLETE signal 206 for receipt by the PLC 120 at step 
117, and, the PLC will generate a suitable control sig- 
nal 209 to enable the engagement jaws of the preci- 

30 sion conveyor engagement boat 40 to engage the 
needle at step 119. In the preferred embodiment, the 
dwell time of the NEEDLE PLACE COMPLETE signal 
206 is approximately 48-64 milliseconds. After acti- 
vating this signal, the robot assembly 50a,b will hold 

35 the needle in place for the same time period. (48-64 
msec.) Immediately thereafter, the robot will open its 
grippers and move back to its approach location away 
from the engagement boat 40, as indicated as step 

121 in Figure 8(b). Finally, the DONT INDEX PRECI- 
40 SION CONVEYOR signal 204 is removed at step 1 23 

indicating that it is now clear for the PLC (and Con- 
veyor Control task) to initiate the indexing of the pre- 
cision conveyor 35 which is performed at the com- 
mand of the PLC 120 at step 125 in Figure 8(b). 

45 Before indexing the As a safety interlock for con- 

veyor index initiation, the Robot Control Task 150 will 
make a determination as to whether the current nee- 
dle picked up is the last needle of the maximum num- 
ber of (three) needles to be recognized in a camera's 

so FOV and placed in the robot FIFO. This step is indi- 
cated as step 132 in Figure 8(c). If the current needle 
picked is the last needle, the Robot task 150 for the 
current Adept will signal the Conveyor Indexing Con- 
trol Task 180 with an internal control LAST PICK sig- 

55 nal 192, 196 indicating that the respective robot as- 
sembly, 50a or 50b, has picked up the last needle 
from the current conveyor as indicated as step 1 34 in 
Figure 8(c). If the maximum number of needles ex- 
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pected per camera f ield-of-view ("FONT) is not picked 
from the respective current infeed conveyor belt 
25a(,b), e.g., only two needle locations were placed 
in the FIFO buffer as indicated at step 1 35 in Fig. 8(c), 
the Robot Control Task 150 will request the Conveyor 
Control task 1 80 to index that conveyor belt "early" via 
the INDEX CONVEYOR 1 EARLY or the INDEX CON- 
VEYOR 2 EARLY signals 211,212, respectively, as 
shown in Figure 7, and indicated as step 136 in Figure 
8(c). 

Since all signals affecting the motion of the con- 
veyors are routed through the Conveyor Control task 
1 80, the Conveyor Control task will generate a corre- 
sponding INDEX CONVEYOR 1 EARLY, signal 
2ir or INDEX CONVEYOR 2 EARLY, signal 212\ for 
receipt by the other Adept robot If during normal op- 
eration a Robot Control Task receives either Index 
Conveyor 1 Early or the Index Conveyor 2 Early sig- 
nal, it will flush the contents of its FIFO buffer 155 and 
continue as if the last needle has been picked from 
the conveyor. 

As a result of receiving the INDEX CONVEYOR 

1 EARLY, signal 21V or INDEX CONVEYOR 2 EAR- 
LY, signal 21 2' from the Conveyor Control task 1 80 in- 
dicating that the maximum number of needles have 
not been picked up or that there are no or insufficient 
needles in the respective camera's FOV, the other 
Adept robot will generate a corresponding CONVEY- 
OR 1 INDEXED EARLY, signal 198', or CONVEYOR 

2 INDEXED EARLY signal 1 99' for receipt by the Con- 
veyor Control task 180, as shown in Figure 7. These 
signals will cause the corresponding conveyor 
25a(,b) to abort processing and initiate indexing of the 
belt 

The control software must take into account the 
floating 1 6-32 ms duration of a digital output based on 
the time slicing of V/V+. This will affect the calculation 
for minimum time required for placement in conjunc- 
tion with setting and resetting the Don't Index Preci- 
sion conveyor signal 204. 

The Robot Control Task 150 performs error re- 
covery on two type of errors. These errors are group- 
ed as indexing errors and gross errors. As in all other 
tasks, gross errors cause the Task Manager 240 error 
recovery to respond and stop the Robot Control Task 
immediately. An indexing error occurs if a robot is 
waiting for a needle to be placed in its parts FIFO and 
both conveyor belts have not indexed within an ap- 
propriate amount of time. The Robot Control Task 1 50 
recovers from this type of error by requesting the 
other robot to index early via INDEX CONVEYOR 1 
EARLY (signai 211) or INDEX CONVEYOR 2 EARLY 
(signal 212). This forces both vision/robot control sys- 
tems to flush the contents of its current parts FIFO 
and index the conveyor belts. 



Conveyor Indexing Control Task 

The Conveyor Indexing Control Task 180 initiates 
the indexing of each respective translucent indexing 

5 conveyor 25a,b and the task is initiated by the Con- 
veyor Initiation task 1 90. All signals affecting the mo- 
tion of the conveyors are routed through the Convey- 
or Control task 180, the flow diagram of which is illu- 
strated in Figure 8(d). 

w As shown in Figure 8(d), and in further view of 
Figure 7, the first step 141 of the Conveyor Indexing 
Control task 1 80 is to check for the LAST PICK signal 
192,196 internally generated from the Robot Control 
Task 150 and indicating that the last needle pick-up 

15 from the respective infeed translucent conveyor 
25a,25b has been completed by one of the Adept® ro- 
bots 50a,b. For e.g., at step 142 in Figure 8(d) the de- 
termination is made as to whether the Adept robot 50a 
(Robot I) has initiated the LAST PICK signal from 

20 Conveyor I (25a) or Conveyor II (25b). As a result of 
receiving the LAST PICK signals 1 92,1 96 from the ro- 
bot task, the Conveyor Control task will generate a 
corresponding INDEX CONVEYOR 1 signal 198, or, 
an INDEX CONVEYOR 2 signal 1 99, for receipt by the 

25 PLC 120. This is indicated as respective steps 143 
and 143' in Figure 8(d). It is essential that each 
Adept® robot controller must request the PLC 120 to 
index a translucent indexing conveyor 25a(,b) after 
the last needle has been picked from the respective 

30 conveyor. Therefor, the other Adept® robot must gen- 
erate its corresponding INDEX CONVEYOR 1 (or IN- 
DEX CONVEYOR 2) signal for receipt by the PLC at 
respective steps 144 and 144' before it can command 
the current translucent conveyor 25a, (25b) to index. 

35 Only after receipt of both INDEX CONVEYOR 1 (or 
INDEX CONVEYOR 2) signals 198,199 from each of 
the robot assemblies, as shown at steps 146 and 
146', the PLC 120 commands the translucent index- 
ing conveyor 25a to index, and, at steps 148 and 148', 

40 generates a corresponding CONVEYOR 1 SETTLED 
signal 241 or, a CONVEYOR 2 SETTLED signal 242 
for receipt by the Conveyor Control Task 180. Note 
that the CONVEYOR 1 SETTLED signal 241 and the 
CONVEYOR 2 SETTLED signal 242 are raised ap- 

45 proximately 2 seconds after the PLC has been re- 
quested by the robot control task 1 50 to index convey- 
or 25a, (25b). The Conveyor Control Task 180 then in- 
forms the Vision Control task 160 to begin needle im- 
aging upon receipt of internal control signals 

50 241', 242' that correspond to the respective CON- 
VEYOR 1 SETTLED or the CONVEYOR 2 SETTLED 
signals 241,242. Once the indexing conveyor 25a 
(25b) has been indexed and the corresponding CON- 
VEYOR SETTLED signal 241 ,242 has been received, 

55 the Vision Control Task 1 60 may begin needle recog- 
nition in the corresponding cameras's FOV, as indi- 
cated at step 1 51 and 1 51 ' in Figure 8(d). Specifically, 
under the control of the Vision Control task 160, the 
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cameras 62,64 of the recently indexed conveyor 
25a(,b) will take a snapshot of the respective field of 
views at illuminated portions 30a v b thereof, and the 
task will process the image to make a determination 
of whether recognizable needles are present in each 
camera's field of view as indicated at step 153 in Fig- 
ure 8(d). 

At this point, a distinction must be made between 
the mere presence or detection of a needle in the field 
of view and the presence of a "recognizable" needle. 
A needle may be present, but, for a variety of reasons, 
the Vision Task 160 may not be able to determine its 
positional coordinates until the camera vision para- 
meters are changed by the execution of an auto- 
imaging algorithm which automatically adjusts the iris 
and vision system lighting parameters of each camera 
so that the cameras may subsequently obtain en- 
hanced images that may be processed. During steady 
state, when the vision task has already "recognized" 
a needle in its respective field of view, the auto- 
imaging algorithm is not repeated. Details of the auto- 
imaging algorithm will be explained in detail below. 

As an alternative to receiving the LAST PICK sig- 
nals, the Conveyor Indexing Control task 1 80 may re- 
ceive the INDEX CONVEYOR EARLY (1 and 2) sig- 
nals 231,232 internally generated from the Vision 
Control task 160 when no needles are recognized in 
the current camera FOV, or, may receive the INDEX 
CONVEYOR EARLY (1 and 2) signals 211,212 inter- 
nally generated from the Robot Control task 150 
when the maximum number of needles are not 
picked. In either case, the Conveyor Control task 180 
performs the following procedures as illustrated in 
the flow diagram of Figure 8(e). 

As indicated at step 157 in Figure 8(e), the Con- 
veyor Control task has received an INDEX CONVEY- 
OR 1 EARLY signal, for e.g., signal 211 from Robot 1 
of the Robot Control task 150 indicating that the max- 
imum number of needles could not be picked from the 
Conveyor 1. The Conveyor Control task immediately 
generates a corresponding INDEX CONVEYOR 1 
EARLY signal (211 ') for receipt by the Other Adept Ro- 
bot as indicated at step 159. This signal informs the 
other Adept robot to stop processing Conveyor 1 nee- 
dles and to index the belt. The other Adept robot will 
respond by generating a corresponding CONVEYOR 
1 INDEXED EARLY signal (198') as indicated at step 
1 61 in Figure 8(e) to inform the Conveyor Control task 
1 80 that the other Adept (e.g., Robot 2) will abort con- 
veyor 1 processing and will index the conveyor 1 early 
as requested by the first Adept (Robot 1). Once these 
signals are received by Conveyor Control task, the 
task will immediately generate the INDEX CONVEY- 
OR 1 signal (198) for receipt by the PLC which will ini- 
tiate the early indexing of the requested conveyor 
belt, e.g. conveyor 1 (25a) as indicated in Figure 8(d). 
The execution of the Vision Control task 160 will fol- 
low thereafter upon receipt of the CONVEYOR 1 



SETTLED signal. 
Vision Control Task 

s The Vision Control Task 160 controls and proc- 

esses the images taken by each of the two camera 
assemblies 62,64. Since the timing of the two trans- 
lucent conveyors are phased, only one camera is op- 
erating at one time. 

w Specifically, as shown in Figure 3(b), the Vision 
Control task 160 interfaces with each camera 62,64 
to identify the needle locations of recognizable nee- 
dles in that camera lens's respective field of view en- 
compassing an area located at respective illuminated 

15 platforms 30a,30b. The Vision Task 160 then proc- 
esses the positional and orientation information of the 
identified needle locations and writes those locations 
to the Robot Task FIFO 155 via data lines 197. As 
mentioned above, the Vision Control task is addition- 

20 ally responsible for initiating an early conveyor index 
if no needles were imaged in a camera field of view. 

As described briefly above, the Vision Control 
task runs each time either conveyor 25a,25b com- 
pletes indexing. It is initiated to begin needle recogni- 

25 tion upon receipt of either internally generated CON- 
VEYOR 1 SETTLED signal 241' or CONVEYOR 2 
SETTLED signal 242' which is generated by the PLC 
120 and routed through the Conveyor Control task 
1 80 each time respective translucent indexing con- 

30 veyor 25a ,25b has ceased indexing, as commanded 
by the Adepts. Each CONVEYOR SETTLED signal 
241,242 goes high (logic "1") approximately two (2) 
seconds after the PLC has been requested by the 
Adept® robot to index a translucent indexing convey- 

35 or. Each of the CONVEYOR SETTLED signals 1 and 
2 (241,242) remain high until the PLC 120 receives 
the next respective INDEX CONVEYOR 1 or 2 signal 
1 98, 1 99 from the Adept robots. 

As illustrated in Figure 8(f), the Vision Task 160 

40 activates that camera which is associated with the 
conveyor settled signal. When activated, the camera 
62,64 takes a picture of the backlit areas 30a,b of the 
conveyor belt 25a,(25b) as indicated at step 301 . Any 
image obtained is preferably converted to binary im- 

45 age data as indicated at step 303 for subsequent dig- 
ital processing, indicated at step 305. The Vision Con- 
trol task 160 utilizes 'Vision tools" to detect accept- 
able needles, and places the coordinates of accept- 
able needle pick-up points in the FIFO buffer 155 for 

so the Robot task. An "acceptable" needle in the backlit 
areas is a needle that measures within the tolerances 
of the needle parameters that have been previously 
accepted during the needle changeover procedure. 
The needle changeover procedure is a procedure to 

55 inform the infeed system software of the type and 
size of the needles in the current batch to be process- 
ed and must be executed before making needle batch 
changes as to be discussed below. Specified needle 
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tolerances are for the needle radius, barrel width, an- 
gular characteristics of the needle with respect to the 
robots, and the calculated area as computed from the 
needle parameters. 

5 

Auto-Imaging Algorithm 

As mentioned above, if a detected needle is un- 
recognizable, the auto-imaging algorithm is invoked 
to change the camera vision parameters. Thus, after 10 
the binary image data is processed at step 305 as 
shown in Figure 8(h), a determination is made as to 
whether the needle image is of the specified radius 
(step 307), whether the needle image is of the speci- 
fied barrel width (step 309), whether the needle im- is 
age has the specified angular characteristics (step 
311), and, whether the needle image area is within the 
specified tolerance (step 313). If any of these criteria 
are out of specification, then the auto-imaging algo- 
rithm is executed at step 315. The function of the 20 
auto- imaging procedure is to take a series of pictures 
of the same needle image at the respective camera's 
field of view to enhance the needle image for better 
needle recognition by improving the vision parame- 
ters between pictures. Thus, after each of the series 25 
of pictures is taken, the auto-imaging algorithm will 
automatically adjust the camera's iris and vision sys- 
tem lighting parameters to enable the vision system 
to image the needles properly within the camera's 
field of view. For example, when adjusting the lighting 30 
of the fields of view, certain camera vision parameters 
such as the gain, offset, and binary threshold may be 
modified. The auto-imaging algorithm is executed un- 
til a needle is recognized in each camera's field of 
view and is not repeated until a needle changeover is 35 
executed. 

Even when the cameras of the Vision Control 
task 160 are adjusted, needle images may still not be 
imaged properly. This is because each camera's field 
of view utilizes a backlighting source and needles that 40 
5 overlap, touch with each other, or, are clipped by 
field of view edge boundaries will not be considered 
for recognition. Thus, as indicated in Figure 8(h) at 
step 319, the Vision Control task will make a determi- 
nation of whether the needles overlap or touch each 45 
other, and, at step 321, will determine whether the 
needles are too close to the edge of the field of view. 

After all of the possible needles are recognized, 
the Vision Control task will calculate the needle pick- 
up coordinates of the acceptable needles and place so 
them in the Robot Control task FIFO buffer 1 55 to en- 
able the robot to pick and place the acceptable needle 
onto the precision conveyor, as indicated at step 323. 
In the preferred embodiment, the maximum number 
of needles that can be recognized during each dwell 55 
cycle of each translucent indexing conveyor is three 
(3). If less than this maximum or if no needles are rec- 
ognized, a robot may be signalled to index the corre- 
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sponding conveyor early, causing the vision system 
to abort its processing as described above. 

Vision Task 160 is responsible for limiting the 
number of needle locations written to the FIFO to 
three, since the Robot Control Task will pick and place 
a needle for every needle location passed to the FIFO 
155. In the preferred embodiment, the Vision Task is 
limited to operate for five seconds per indexing con- 
veyor cycle. 

The Vision Control Task 160 performs error re- 
covery on three types of errors. These errors are 
grouped as imaging errors, processing errors, and 
gross errors. The gross errors cause the Task Man- 
ager error recovery to respond and stops the Vision 
Control Task 160 immediately. When an imaging er- 
ror occurs, the Vision Control Task 160 suspends all 
execution on the current FOV and requests an early 
index of the conveyor belt by generating either INDEX 
CONVEYOR 1 EARLY or INDEX CONVEYOR 2 EAR- 
LY signals 231,233 as discussed above. Receipt of 
these signals causes no needles to be placed in the 
parts FIFO and forces both vision/robot systems to 
pass on the current FOV of needles. If a processing 
error occurs, the Vision Control Task suspends all 
processing on the current needle and begins process- 
ing a new needle in the same FOV if another needle 
is available. As a result, the Vision Task does not in- 
sert the needle into the parts FIFO. 

Conveyor Initiation Task 

The Conveyor Initiation Task 190functions to ini- 
tiate the Conveyor Indexing Control task 180 and is 
started whenever the ROBOT ENABLE signal 219 is 
raised from the PLC 120. Once started, this task re- 
quests an INDEX INFEED CONVEYOR 1 (25a), sig- 
nal 237, then waits approximately two (2) seconds, 
and requests an INDEX INFEED CONVEYOR 2 
(25b), signal 239, as shown in Figure 7. The task 190 
is then terminated and is not restarted again until the 
ROBOT ENABLE signal 219 is lowered and raised 
again. 

Task Manager 

The Task Manager 240 initializes the software 
and hardware I/O signals, the global variables, and 
the vision/robot system tasks. Once the vision/robot 
system tasks are running, the task manager monitors 
the integrity and status of each task currently running 
and the resources that are controlled by these tasks. 
The status poll signals 247a- 247f are indicated in Fig- 
ure 7. The resources are the robot, communication 
ports, and the I/O signal lines. The Task Manager re- 
ports any errors to the PLC, via the SYSTEM FAIL 
signal 222, and the SCADA node, via the SCADA 
Node Interface Task 195. The SYSTEM FAIL signal 
222 is generated whenever a robot (as detected by 
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the Task Manager) has recognized a gross error 
which prevents it from continuing operation. This sig- 
nal is active-low and remains low until the Adept robot 
is reset Thus, the PLC must lower the ROBOT EN- 
ABLE signal 219 immediately upon receiving this sig- 
nal. 

For gross errors occurring with the vision/robot 
control software, the Task Manager 240 is utilized to 
detect and recover from these errors by continuously 
polling the status arid integrity of all steady-state 
tasks and resources during program execution. If it is 
determined that a gross error has occurred, the SYS- 
TEM FAIL signal 222 will be raised to the PLC 120 and 
all tasks except the SCADA Node Interface Task, the 
Control Panel Task and the Task Manager will be 
stopped. A code indicating the reason for the last un- 
recoverable error will be available to the SCADA 
Node through the SCADA Node Interface Task. In 
some cases, an error message will be displayed in the 
Monitor Window of the Adept robot controller. After 
the SYSTEM FAIL signal is raised, the Task Manager 
will attempt to correct any problems detected on the 
robot and notify the operator through the Monitor 
Window. In most cases, the operator will only need to 
raise the ROBOT ENABLE signal again to re-set the 
vision/robot control software. 

Control Panel Task 

The Control Panel Task 260 presents a mouse 
controlled panel that allows an operator to access va- 
rious software "debugging" utilities, to access diag- 
nostics utilities, to control the speed of the robot, and 
to select new positions that the robot will move to for 
picking and placing needles. Also, the Control Panel 
Task allows the operator to stop the vision/robot sys- 
tem tasks from executing. 

SCADA Node Interface task 

The SCADA Node Interface task 195 polls the 
SCADA Node RS-232 interface for messages from 
the SCADA node, the task will act as slave to SCADA 
Node requests for Adept and camera set-up proce- 
dures necessitated by product changeovers. These 
requests are valid only when the ROBOT ENABLE 
signal 219 is deactivated. 

Lens Control Task 

The Lens Control Task 270 is initiated only when 
the SCADA node requests a new product to be intro- 
duced to the vision system and is executed only as an 
off-line process. The Lens Control Task 270 accepts 
the new needle parameters and adjusts the f ield-of- 
view size for both cameras to accommodate the new 
product size. The zoom, focus, and iris lenses are af- 
fected by this new product introduction, as well as in- 



ternal vision system parameters, such as gain, binary 
threshold, and offset, used for imaging. Once the 
cameras are adjusted, the task is suspended until an- 
other new product is introduced to the vision/robot 
5 system. 

Product Changeover 

Prior to enabling the robots to begin the needle 

10 infeed process, a Needle Changeover procedure is in- 
voked to inform the Vision and Robot Control tasks of 
the control system software of the type and size of 
the needles to be processed. This needle changeover 
procedure must be completed before making needle 

15 batch changes. If a changeover is not completed be- 
fore the first needle batch run after power-up, an error 
message will be displayed at the FIX/DMACS (SCA- 
DA Node) screen when the robots are enabled and 
the robots will not run. If a changeover is not complet- 

20 ed between different needle batch runs, the vision 
tasks will not identify any needle being run. 

Essentially, an operator of the system enters the 
needle parameters in appropriate units, e.g., millime- 
ters and degrees at the FIX/DMACS screen of the 

25 SCADA task 195 through data lines 229. Such needle 
parameters for use by the Vision tasks include, the 
needle radius and the radius tolerance, acceptable 
needle angles and their tolerances, and, the needle 
width and the width tolerance. 

30 In addition to inputting needle change parame- 

ters for the vision tasks, initial camera set-up parame- 
ters associated with the particular batch of needles to 
be processed are also input through the SCADANode 
for use by the system. As shown in Figure 7, the sof t- 

35 ware utilizes the information provided by the user via 
the SCADA Node to automatically adjust the lens for 
the correct f ield-of-view size, focus, and zoom para- 
meters prior to enabling the robots. 

Figures 5(a) - 5(c) illustrate the precision convey- 

40 or boat 40 to which each needle 19 is transferred by 
the robots. Each boat is preferably provided with a 
pair of jaws; one jaw 47 being fixedly mounted, and 
the second jaw 49 being slidable within pocket 42. In 
operation, a push rod 46 is pressed in the direction of 

45 the arrow "A" shown in Figure 5(c) to compress spring 
52 which retracts the position of the movable jaw 49 
in the direction indicated by the arrow W B" to allow for 
placement of needle 19 within the notch 44 of both 
jaws. Normally, spring 52 is biased as shown in Figure 

so 5(b) to maintain movable jaw 49 in its engaged posi- 
tion for retaining a needle 19 in the notch 44. It should 
be understood that any type of releasable engaging 
mechanism may be provided for releasably retaining 
a needle 19 on conveyor boat 40, provided that each 

55 needle be correctly oriented on its respective boat for 
subsequent swaging to take place. 

Figure 6 illustrates a robot load solenoid mecha- 
nism 70 that is activated by the PLC 120 each time a 

10 
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needle 1 9 is being transferred to a precision conveyor 
boat 40 as described with respect to step 113 of Fig- 
ure 8(b). The robot load solenoid 70 may be mounted 
to the precision conveyor by an appropriate mounting 
plate 72. Asensormounted on the precision conveyor, 
is also provided to sense the proximity of the preci- 
sion conveyor boat 40. At such time a conveyor boat 
is dwelled for transference of a needle 19 thereto, a 
release arm 56 of the robot load solenoid is actuated 
by solenoid 70 at the initiation of the PLC 1 20 to pivot 
about pin 51 to depress push rod 46 and retract the 
movable jaw 49 to the position illustrated in Fig. 5(c). 
The robot gripper 55a, b then positions the needle 19 
between the jaws 47,49 of conveyor boat 40 for en- 
gagement thereof. The release arm 56 is then retract- 
ed by spring 78 as the conveyor boat 40 resumes 
movement as initiated by the PLC 120 (See Step 113 
Figure 8(b)). 

For automatic swaging to take place at the swag- 
ing station it is necessary that the needle be precisely 
positioned within the notch 44 of engagement jaws 
47,49 of the boat 40. This is because the multi-axis 
gripper generally indicated at step 17 in the system 
flow chart of Figure 1 , must receive a precisely posi- 
tioned needle for a suture (not shown) to be placed 
within the end 5 of needle 19. To ensure that each 
needle is uniformly oriented for transference to the 
multi-axis gripper of the automatic swaging station, a 
needle orientation device ("plow") 54 is provided as 
shown in Figures 9(a) - 9(e) to orient each needle 
while engaged between jaws 47,49 on conveyor boat 
40. The plow comprises an elongated arcuate blade 
57 protruding from a mounting bracket 58 as best 
shown in Figures 9(a) and 9(b). In the preferred em- 
bodiment shown in Figure 9(c), the plow is fixedly 
mounted at one end 48 of the precision conveyor 35 
to enable arcuate blade 57 to scoop needle 19 posi- 
tioned on the conveyor boat 40 while in forward mo- 
tion. After contact is made, the arcuate portion 87 of 
the needle 19 is lifted and rolls over the arcuate blade 
57 of the plow 54 as shown in Figures 9(c) through 
9(e). Provision of the plow 54 ensures that each nee- 
dle conveyed to the suture swaging station is oriented 
in the same direction. 

Another mechanism is provided for further orient- 
ing the needle upon the precision conveyor boat is the 
needle hard stop assembly 95 illustrated in Figures 
10(a) and 10(b). The hard stop assembly 95 compris- 
es a pulley 99 operable by a drive motor (not shown) 
and timing belt 97 for rotating a cam 98 as shown in 
Fig. 10(a). Cam follower 91 is provided for actuating 
arm stop 93 to reciprocate from a first position above 
the engagement jaws 47,49 of conveyor boat 40, to a 
position that enables blade 94 of arm stop 93 to bear 
upon the end 85 of needle 19 while the precision con- 
veyor boat 40 is conveyed in the forward direction as 
indicated by the arrow in Fig. 10(b). Impeding the for- 
ward motion of the needle 19 by blade 94 forces the 



needle to move within engagement jaws 47,49 of the 
conveyor boat 40 so that the engagement jaws 47,49 
engage the needle at a precise location, for e.g., its 
barrel portion 83. Note that the cam 98, as driven by 

5 timing belt 97, is designed so that the arm stop 93 re- 
ciprocates in a timed relation with the forward motion 
of the boat 40 so that each needle upon each convey- 
or boat 40 is further oriented. After the needle is ori- 
ented, the arm stop 93 is reciprocated to its position 

w above the conveyor boat 40 to await the next needle 
for further orientation in the manner heretofore de- 
scribed. 

After the precision conveyor boat 40 is equipped 
with a properly oriented needle 19 in the manner do- 
ts scribed above, it is conveyed to an automatic swaging 
station (not shown) where a suture is fixedly attached 
to the needle. A stop assembly 80, shown in Figures 
11(a) and 11(b), is the mechanism for executing a 
hard stop of the needle carrying conveyor boat 40 
20 when the boat has reached the end of its destination 
at the needle swaging station. The blade 82 of the 
hard stop assembly 80 provides a fine tuning of the 
position of the needle upon the boat 40. Specifically, 
the blade 82 orients the needle to within 0.001 inches 
25 of the final position required for automatic swaging to 
take place. 

4 While the invention has been particularly shown 
and described with respect to the preferred embodi- 
ments thereof, it will be understood by those skilled 
30 in the art that the foregoing and other changes in form 
and details may be made therein without departing 
from the spirit and scope of the invention, which 
should be limited only by the scope of the appended 
claims. 

35 

Claims 

1 . A control system for a needle infeed device for au- 
40 tomatically transferring needles randomly posi- 

tioned on a further processing location, said nee- 
dle infeed device comprising one or more robots 
each having a gripper means for picking and plac- 
ing said needles, said control system comprising: 
45 (a) control means for pausing said conveyor 

means to create a dwell cycle for said infeed 
device; 

(b) vision tracking means in communication 
with said control means for imaging needles 

50 at one or more predetermined locations on 

said conveyor means and for determining 
positional and orientation data for each im- 
aged needle during said dwell cycle; 

(c) memory means for temporarily storing said 
55 positional and orientation data received from 

said vision tracking means; and 

(d) robot control means for accessing said 
stored positional and orientation data corre- 
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6. 



sponding to said imaged needles from said 
memory means, and enabling said one of said 
robots to pick up said imaged needle in accor- 
dance with its respective positional and orien- 
tation data and place said needle in said en- s 
gagement device. 

The control system for a needle infeed device as 
claimed in Claim 1 wherein said needle infeed de- 
vice includes a drive means for driving said con- 10 
veyor means, said robot control means generat- 
ing a first signal for receipt by said control means 
requesting said control means to enable said 
drive means to index said conveyor means. 

15 

The control system for a needle infeed device as 
claimed in Claim 2 wherein said control means 
generates a first signal for receipt by said vision 
tracking means indicating that said conveyor 
means has finished indexing and is in said dwell 20 
cycle. 

The control system for a needle infeed device as 
claimed in Claim 1 wherein said vision tracking 
means includes one or more camera means for 25 
obtaining a video image of said needles on said 
conveyor means at each of respective said one or 
more predetermined locations within a field-of- 
view of each of said one or more cameras. 

30 

The control system for a needle infeed device as 
claimed in Claim 4 wherein each of said cameras 
has a plurality of vision parameters associated 
therewith, said vision tracking means including 
means for comparing needle parameters ob- 35 
tained f rom said video image with one or more ac- 
ceptable needle parameters associated with a 
current batch of needles to be processed, said 
one or more needle parameters selected from the 
group including needle radius, needle angle, and 40 
needle width. 

The control system for a needle infeed device as 
claimed in Claim 1 wherein said vision tracking 
means further includes means for automatically 45 
enhancing said image of a needle by recording 
successive images of said needle and adjusting 
one or more of a plurality of vision parameters be- 
tween each successive image until said image of 
said needle is acceptable for obtaining positional 50 
coordinate data therefrom. 

The control system for a needle infeed device as 
claimed in Claim 6 wherein said vision parame- 
ters include field of view size, iris control for said 55 
camera and vision system lighting control for said 
camera. 



8. The control system for a needle infeed device as 
claimed in Claim 2 wherein said robot control 
means automatically polls said memory means to 
obtain current positional and orientation data of 
each recognized needle for said robots, said ro- 
bot control means generating said first signal for 
receipt by said control means to further index 
said conveyor means when no positional and ori- 
entation data is available in said memory means 
at a current dwell cycle. 

9. The control system for a needle infeed device as 
claimed in Claim 1 wherein said positional and 
orientation data includes positional coordinates 
of each recognized needle for said robots, said 
control means enabling said gripper means of 
each said one or more robots to pick up said one 
or more needles at said positional coordinates, 
said robot control means further generating a 
second signal for receipt by said control means 
indicating said one or more needles have been 
picked up by said robot gripper means. 

10. The control system for a needle infeed device as 
claimed in Claim 9 wherein said engagement de- 
vice is located on a second conveyor means hav- 
ing a drive means associated therewith for index- 
ing said needle to said processing location, said 
robot control means generating a third signal for 
receipt by said control means requesting said 
control means to pause said indexing of said sec- 
ond conveyor means to create a second dwell 
time therefor. 

11. The control system for a needle infeed device as 
claimed in Claim 10 wherein said control means 
generates a signal for receipt by said robot con- 
trol means indicating that said robot gripper 
means may place said needle in said engage- 
ment device during said second dwell time. 

12. The control system for a needle infeed device as 
claimed in Claim 11 wherein said robot control 
means generates a fourth signal for receipt by 
said control means indicating that said one or 
more robots have placed said needles in said en- 
gagement devices, said control means enabling 
first and second jaws of said engagement device 
to grip said needles placed therein. 

13. A method for controlling an automatic infeed de- 
vice for feeding surgical needles from one loca- 
tion to another location, said infeed device having 
a first conveyor having randomly positioned nee- 
dles located thereon, a second conveyance hav- 
ing a plurality of needle engagement devices lo- 
cated thereon, and one or more robot means 
each having a gripper means for picking up a nee- 
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die from said indexing conveyor, said method 
comprising the steps of: 

(a) pausing said first conveyor to create a 
dwell time for said infeed device; 

(b) visually tracking said needles on said con- 
veyor using a vision tracking means during 
said dwell time to determine acceptable nee- 
dle locations for said one or more robot 
means; 

(c) picking up said needles at said acceptable 
needle locations with said one or more robot 
gripper means; and, 

(d) placing each needle in one of said needle 
engagement devices for subsequent convey- 
ance thereof. 

14. The method for controlling an automatic infeed 
device for feeding surgical needles from one lo- 
cation to another location according to Claim 13, 
wherein said step (a) of pausing said first convey- 
or further includes the step of generating a first 
control signal from said robot means requesting 
a control means for said first conveyor to inhibit 
motion thereof during said dwell time. 

15. The method for controlling an automatic infeed 
device for feeding surgical needles from one lo- 
cation to another location according to Claim 13, 
wherein said step (b) of visually tracking said nee- 
dles on said conveyor during said dwell time to 
determine acceptable needle locations for said 
one or more robot means further includes the 
steps of: 

(a) generating a signal for said vision tracking 
means indicating that said first conveyor 
means is in said dwell cycle; 

(b) obtaining an image of said needles from 
one or more camera means each having a 
field of view at one or more predetermined lo- 
cations on said conveyor means; 

(c) processing said image to determine posi- 
tional coordinates for recognizable needles 
present in said image; and, 

(d) inputting said positional coordinates into a 
memory means for access by said robot 
means. 

16. The method for controlling an automatic infeed 
device for feeding surgical needles from one lo- 
cation to another location according to Claim 15, 
wherein said processing step (b) further includes 
the steps of: 

(a) determining one or more needle parameter 
values for needles identified in said image, 
said one or more needle parameters selected 
from the group including needle radius, nee- 
dle angle, and needle width; and 

(b) comparing each of said needle parameters 



values obtained from said image with a prede- 
termined range of acceptable needle parame- 
ter values associated with a current batch of 
needles being processed. 

5 

17. The method for controlling an automatic infeed 
device for feeding surgical needles from one lo- 
cation to another location according to Claim 13, 
which further includes an image enhancing step 

10 to calibrate the device, the enhancing step includ- 
ing the steps of: 

(a) successively obtaining a series of images 
of one of said needles during said tracking 
step; and 

15 (b) adjusting one or more of a plurality of cam- 

era vision parameters selected from the group 
including field of view size, iris control for said 
camera, and vision system lighting control for 
said camera, between each successive im- 

20 age until said image of said needle is accept- 

able to obtain data to determine acceptable 
needle locations. 

18. The method for controlling an automatic infeed 
25 device for feeding surgical needles from one lo- 
cation to another location according to Claim 15, 
wherein said processing step (b) further includes 
the step of determining whether needles located 
in said f ield-of-view overlap with each other. 

30 

19. A method for controlling an automatic infeed de- 
vice for feeding surgical needles from one loca- 
tion to another location according to Claim 14 fur- 
ther including the step of generating a second 

35 control signal from said robot means for said con- 
trol means requesting said control means to in- 
dex said first conveyor means when no accept- 
able needle locations are available. 

40 20. The method for controlling an automatic infeed 
device for feeding surgical needles from one lo- 
cation to another location according to Claim 1 3, 
wherein said step (d) of placing each said needles 
in an engagement device further includes the 
45 steps of: 

(a) pausing said second conveyance having 
said needle engagement devices located 
thereon to create a second dwell time for said 
infeed system; and, 
so (b) generating a control signal indicating to 

said one or more robot gripper means to place 
a gripped needle in said needle engagement 
device during said second dwell time. 

55 21. The method for controlling an automatic infeed 
device for feeding surgical needles from one lo- 
cation to another location according to Claim 20, 
said method further including the step of gener- 
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ating a signal for enabling a pair of jaws of said 
engagement device to grip said needle after 
placement therein by said robot gripper means. 

22. The method for controlling an automatic infeed 5 
device for feeding surgical needles from one lo- 
cation to another location according to Claim 20 
further including the step of actuating a push rod 
means for retracting one jaw of said pair of en- 
gagement jaws for enabling the positioning of 10 
said needle therebetween, said actuating step 
occurring prior to the placement of said needle 
between said pair of engagement jaws. 

23. The method for controlling an automatic infeed 15 
device for feeding surgical needles from one lo- 
cation to another location according to Claim 20 
further including the step of orienting said needle 
while positioned upon said second conveyor 
means. 20 
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FIG. 4 
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FIG. 5 A 
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